<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/xhtml;charset=UTF-8"/>
<meta http-equiv="X-UA-Compatible" content="IE=9"/>
<meta name="generator" content="Doxygen 1.8.7"/>
<title>Parallel Gaussian Process Regression: src/pgpr_plma.h Source File</title>
<link href="tabs.css" rel="stylesheet" type="text/css"/>
<script type="text/javascript" src="jquery.js"></script>
<script type="text/javascript" src="dynsections.js"></script>
<link href="search/search.css" rel="stylesheet" type="text/css"/>
<script type="text/javascript" src="search/search.js"></script>
<script type="text/javascript">
  $(document).ready(function() { searchBox.OnSelectItem(0); });
</script>
<link href="doxygen.css" rel="stylesheet" type="text/css" />
</head>
<body>
<div id="top"><!-- do not remove this div, it is closed by doxygen! -->
<div id="titlearea">
<table cellspacing="0" cellpadding="0">
 <tbody>
 <tr style="height: 56px;">
  <td style="padding-left: 0.5em;">
   <div id="projectname">Parallel Gaussian Process Regression
   &#160;<span id="projectnumber">1.0.0</span>
   </div>
   <div id="projectbrief">The implementation of parallel Gaussian process (GP) regression is based on the following publication: Jie Chen, Nannan Cao, Kian Hsiang Low, Ruofei Ouyang, Colin Keng-Yan Tan &amp; Patrick Jaillet. Parallel Gaussian Process Regression with Low-Rank Covariance Matrix Approximations. In Proceedings of the 29th Conference on Uncertainty in Artificial Intelligence (UAI 2013), Bellevue, WA, Jul 11-15, 2013.</div>
  </td>
 </tr>
 </tbody>
</table>
</div>
<!-- end header part -->
<!-- Generated by Doxygen 1.8.7 -->
<script type="text/javascript">
var searchBox = new SearchBox("searchBox", "search",false,'Search');
</script>
  <div id="navrow1" class="tabs">
    <ul class="tablist">
      <li><a href="index.html"><span>Main&#160;Page</span></a></li>
      <li><a href="pages.html"><span>Related&#160;Pages</span></a></li>
      <li><a href="annotated.html"><span>Classes</span></a></li>
      <li class="current"><a href="files.html"><span>Files</span></a></li>
      <li>
        <div id="MSearchBox" class="MSearchBoxInactive">
        <span class="left">
          <img id="MSearchSelect" src="search/mag_sel.png"
               onmouseover="return searchBox.OnSearchSelectShow()"
               onmouseout="return searchBox.OnSearchSelectHide()"
               alt=""/>
          <input type="text" id="MSearchField" value="Search" accesskey="S"
               onfocus="searchBox.OnSearchFieldFocus(true)" 
               onblur="searchBox.OnSearchFieldFocus(false)" 
               onkeyup="searchBox.OnSearchFieldChange(event)"/>
          </span><span class="right">
            <a id="MSearchClose" href="javascript:searchBox.CloseResultsWindow()"><img id="MSearchCloseImg" border="0" src="search/close.png" alt=""/></a>
          </span>
        </div>
      </li>
    </ul>
  </div>
  <div id="navrow2" class="tabs2">
    <ul class="tablist">
      <li><a href="files.html"><span>File&#160;List</span></a></li>
    </ul>
  </div>
<!-- window showing the filter options -->
<div id="MSearchSelectWindow"
     onmouseover="return searchBox.OnSearchSelectShow()"
     onmouseout="return searchBox.OnSearchSelectHide()"
     onkeydown="return searchBox.OnSearchSelectKey(event)">
<a class="SelectItem" href="javascript:void(0)" onclick="searchBox.OnSelectItem(0)"><span class="SelectionMark">&#160;</span>All</a><a class="SelectItem" href="javascript:void(0)" onclick="searchBox.OnSelectItem(1)"><span class="SelectionMark">&#160;</span>Classes</a><a class="SelectItem" href="javascript:void(0)" onclick="searchBox.OnSelectItem(2)"><span class="SelectionMark">&#160;</span>Files</a><a class="SelectItem" href="javascript:void(0)" onclick="searchBox.OnSelectItem(3)"><span class="SelectionMark">&#160;</span>Functions</a><a class="SelectItem" href="javascript:void(0)" onclick="searchBox.OnSelectItem(4)"><span class="SelectionMark">&#160;</span>Variables</a><a class="SelectItem" href="javascript:void(0)" onclick="searchBox.OnSelectItem(5)"><span class="SelectionMark">&#160;</span>Pages</a></div>

<!-- iframe showing the search results (closed by default) -->
<div id="MSearchResultsWindow">
<iframe src="javascript:void(0)" frameborder="0" 
        name="MSearchResults" id="MSearchResults">
</iframe>
</div>

<div id="nav-path" class="navpath">
  <ul>
<li class="navelem"><a class="el" href="dir_68267d1309a1af8e8297ef4c3efbcdba.html">src</a></li>  </ul>
</div>
</div><!-- top -->
<div class="header">
  <div class="headertitle">
<div class="title">pgpr_plma.h</div>  </div>
</div><!--header-->
<div class="contents">
<a href="pgpr__plma_8h.html">Go to the documentation of this file.</a><div class="fragment"><div class="line"><a name="l00001"></a><span class="lineno">    1</span>&#160;<span class="comment">// Copyright (c) 2014, Jiangbo Yu</span></div>
<div class="line"><a name="l00002"></a><span class="lineno">    2</span>&#160;</div>
<div class="line"><a name="l00003"></a><span class="lineno">    3</span>&#160;<span class="comment">// This Source Code Form is subject to the terms of the Mozilla Public</span></div>
<div class="line"><a name="l00004"></a><span class="lineno">    4</span>&#160;<span class="comment">// License, v. 2.0. If a copy of the MPL was not distributed with this</span></div>
<div class="line"><a name="l00005"></a><span class="lineno">    5</span>&#160;<span class="comment">// file, You can obtain one at http://mozilla.org/MPL/2.0/.</span></div>
<div class="line"><a name="l00006"></a><span class="lineno">    6</span>&#160;</div>
<div class="line"><a name="l00012"></a><span class="lineno">   12</span>&#160;<span class="preprocessor">#ifndef PGPR_PLMA_H_</span></div>
<div class="line"><a name="l00013"></a><span class="lineno">   13</span>&#160;<span class="preprocessor">#define PGPR_PLMA_H_</span></div>
<div class="line"><a name="l00014"></a><span class="lineno">   14</span>&#160;<span class="preprocessor">#include &lt;Eigen/Dense&gt;</span></div>
<div class="line"><a name="l00015"></a><span class="lineno">   15</span>&#160;<span class="preprocessor">#include &quot;pgpr_parallel.h&quot;</span></div>
<div class="line"><a name="l00016"></a><span class="lineno">   16</span>&#160;<span class="preprocessor">#include &quot;mpi.h&quot;</span></div>
<div class="line"><a name="l00017"></a><span class="lineno">   17</span>&#160;<span class="preprocessor">#include &quot;<a class="code" href="pgpr__type_8h.html">pgpr_type.h</a>&quot;</span></div>
<div class="line"><a name="l00018"></a><span class="lineno">   18</span>&#160;<span class="preprocessor">#include &quot;<a class="code" href="pgpr__chol_8h.html">pgpr_chol.h</a>&quot;</span></div>
<div class="line"><a name="l00019"></a><span class="lineno">   19</span>&#160;<span class="preprocessor">#include &quot;<a class="code" href="pgpr__cov_8h.html">pgpr_cov.h</a>&quot;</span></div>
<div class="line"><a name="l00020"></a><span class="lineno">   20</span>&#160;</div>
<div class="line"><a name="l00021"></a><span class="lineno">   21</span>&#160;<span class="preprocessor">#include &lt;vector&gt;</span></div>
<div class="line"><a name="l00022"></a><span class="lineno">   22</span>&#160;<span class="keyword">using namespace </span>Eigen;</div>
<div class="line"><a name="l00026"></a><span class="lineno"><a class="line" href="classpgpr__plma.html">   26</a></span>&#160;<span class="keyword">class </span><a class="code" href="classpgpr__plma.html">pgpr_plma</a></div>
<div class="line"><a name="l00027"></a><span class="lineno">   27</span>&#160;{</div>
<div class="line"><a name="l00028"></a><span class="lineno">   28</span>&#160;<span class="keyword">private</span>:</div>
<div class="line"><a name="l00029"></a><span class="lineno">   29</span>&#160;    <a class="code" href="classpgpr__parallel.html">pgpr_parallel</a> parallel;</div>
<div class="line"><a name="l00030"></a><span class="lineno">   30</span>&#160;    MatrixXd localTrainSamples;</div>
<div class="line"><a name="l00031"></a><span class="lineno">   31</span>&#160;    MatrixXd supportSamples;</div>
<div class="line"><a name="l00032"></a><span class="lineno">   32</span>&#160;    vector&lt;VectorXd&gt; *testClusters;</div>
<div class="line"><a name="l00033"></a><span class="lineno">   33</span>&#160;    <a class="code" href="classpgpr__cov.html">pgpr_cov</a> cov;</div>
<div class="line"><a name="l00034"></a><span class="lineno">   34</span>&#160;    <span class="keywordtype">int</span> blocksize;</div>
<div class="line"><a name="l00035"></a><span class="lineno">   35</span>&#160;    <span class="keywordtype">int</span> globalBlocks;</div>
<div class="line"><a name="l00036"></a><span class="lineno">   36</span>&#160;    <span class="keywordtype">int</span> bandwidth;                <span class="comment">// The number of samples should be read by current process</span></div>
<div class="line"><a name="l00037"></a><span class="lineno">   37</span>&#160;    <span class="keywordtype">int</span> localband;</div>
<div class="line"><a name="l00038"></a><span class="lineno">   38</span>&#160;    <span class="keywordtype">double</span> mean;                           <span class="comment">// The mean</span></div>
<div class="line"><a name="l00039"></a><span class="lineno">   39</span>&#160;    <span class="keywordtype">double</span> rmse;                           <span class="comment">// The root mean square error</span></div>
<div class="line"><a name="l00040"></a><span class="lineno">   40</span>&#160;    <span class="keywordtype">double</span> mnlp;                           </div>
<div class="line"><a name="l00041"></a><span class="lineno">   41</span>&#160;    MatrixXd pmu;                        <span class="comment">// equal to a vector, predicted value</span></div>
<div class="line"><a name="l00042"></a><span class="lineno">   42</span>&#160;    <span class="keywordtype">int</span> dim;                             <span class="comment">// Dimension of the data</span></div>
<div class="line"><a name="l00043"></a><span class="lineno">   43</span>&#160;    MatrixXd pvar;                       <span class="comment">// equal to a vector, predicted variance </span></div>
<div class="line"><a name="l00044"></a><span class="lineno">   44</span>&#160;    MatrixXd trueval;                    <span class="comment">// True value</span></div>
<div class="line"><a name="l00045"></a><span class="lineno">   45</span>&#160;    <span class="keywordtype">double</span> elapsed;                        <span class="comment">// Incurring time</span></div>
<div class="line"><a name="l00046"></a><span class="lineno">   46</span>&#160;    <span class="keywordtype">int</span> myid;                            <span class="comment">// The rank of the current process</span></div>
<div class="line"><a name="l00047"></a><span class="lineno">   47</span>&#160;</div>
<div class="line"><a name="l00055"></a><span class="lineno">   55</span>&#160;    <span class="keywordtype">void</span> cluster( <span class="keyword">const</span> MatrixXd &amp; test){</div>
<div class="line"><a name="l00056"></a><span class="lineno">   56</span>&#160;        <span class="keywordtype">double</span> sd = DBL_MAX;</div>
<div class="line"><a name="l00057"></a><span class="lineno">   57</span>&#160;        <span class="keywordtype">double</span> sq_dist = 0;</div>
<div class="line"><a name="l00058"></a><span class="lineno">   58</span>&#160;        <span class="keywordflow">for</span>(<span class="keywordtype">int</span> testid = 0; testid &lt; test.cols(); testid ++){</div>
<div class="line"><a name="l00059"></a><span class="lineno">   59</span>&#160;            <span class="comment">// Compute the local minimal distance                                                                     </span></div>
<div class="line"><a name="l00060"></a><span class="lineno">   60</span>&#160;            <span class="comment">//The first point in the data is the centroid of each cluster                                                    </span></div>
<div class="line"><a name="l00061"></a><span class="lineno">   61</span>&#160;            sd=0;</div>
<div class="line"><a name="l00062"></a><span class="lineno">   62</span>&#160;            <span class="keywordflow">for</span>(<span class="keywordtype">int</span> j = 0 ; j &lt; dim; j ++){</div>
<div class="line"><a name="l00063"></a><span class="lineno">   63</span>&#160;                sd += SQR( localTrainSamples(j, 0) - test(j, testid));</div>
<div class="line"><a name="l00064"></a><span class="lineno">   64</span>&#160;            }</div>
<div class="line"><a name="l00065"></a><span class="lineno">   65</span>&#160;            sd = SQRT(sd);</div>
<div class="line"><a name="l00066"></a><span class="lineno">   66</span>&#160;            <span class="keyword">struct</span>{</div>
<div class="line"><a name="l00067"></a><span class="lineno">   67</span>&#160;                <span class="keywordtype">double</span> value;</div>
<div class="line"><a name="l00068"></a><span class="lineno">   68</span>&#160;                <span class="keywordtype">int</span> virMachineID;</div>
<div class="line"><a name="l00069"></a><span class="lineno">   69</span>&#160;            } in, out;</div>
<div class="line"><a name="l00070"></a><span class="lineno">   70</span>&#160;            in.value = sd;</div>
<div class="line"><a name="l00071"></a><span class="lineno">   71</span>&#160;            in.virMachineID = myid;</div>
<div class="line"><a name="l00072"></a><span class="lineno">   72</span>&#160;            MPI_Barrier(MPI_COMM_WORLD);</div>
<div class="line"><a name="l00073"></a><span class="lineno">   73</span>&#160;            <span class="comment">//use map-reduce method to locate the nearest cluster</span></div>
<div class="line"><a name="l00074"></a><span class="lineno">   74</span>&#160;            MPI::COMM_WORLD.Allreduce(&amp;in, &amp;out, 1, MPI::DOUBLE_INT, MPI::MINLOC);</div>
<div class="line"><a name="l00075"></a><span class="lineno">   75</span>&#160;            testClusters[out.virMachineID].push_back( test.col( testid));</div>
<div class="line"><a name="l00076"></a><span class="lineno">   76</span>&#160;        }</div>
<div class="line"><a name="l00077"></a><span class="lineno">   77</span>&#160;    }</div>
<div class="line"><a name="l00078"></a><span class="lineno">   78</span>&#160;<span class="keyword">public</span>:</div>
<div class="line"><a name="l00087"></a><span class="lineno"><a class="line" href="classpgpr__plma.html#a91e6492d35e220988b68d49cfe851d61">   87</a></span>&#160;     <a class="code" href="classpgpr__plma.html#a91e6492d35e220988b68d49cfe851d61">pgpr_plma</a>(Char *hypf, Char *train, Char * test, Char * supset, <span class="keywordtype">int</span> band, <span class="keywordtype">int</span> blks)</div>
<div class="line"><a name="l00088"></a><span class="lineno">   88</span>&#160;         :globalBlocks(blks), bandwidth(band), parallel(0), cov(hypf)</div>
<div class="line"><a name="l00089"></a><span class="lineno">   89</span>&#160;    {</div>
<div class="line"><a name="l00090"></a><span class="lineno">   90</span>&#160;        <span class="comment">/*Be aware of that the Eigen is optimized using the column-major storeage</span></div>
<div class="line"><a name="l00091"></a><span class="lineno">   91</span>&#160;<span class="comment">         *The matrix should be read in a column-wise way.</span></div>
<div class="line"><a name="l00092"></a><span class="lineno">   92</span>&#160;<span class="comment">         */</span></div>
<div class="line"><a name="l00093"></a><span class="lineno">   93</span>&#160;        mean = cov.mu;</div>
<div class="line"><a name="l00094"></a><span class="lineno">   94</span>&#160;        dim = cov.dim;</div>
<div class="line"><a name="l00095"></a><span class="lineno">   95</span>&#160;        <span class="keywordtype">int</span> dsize = getLines(train);</div>
<div class="line"><a name="l00096"></a><span class="lineno">   96</span>&#160;        <span class="keywordtype">int</span> tsize = getLines(test);</div>
<div class="line"><a name="l00097"></a><span class="lineno">   97</span>&#160;        pmu = MatrixXd::Constant(tsize, 1, 0);</div>
<div class="line"><a name="l00098"></a><span class="lineno">   98</span>&#160;        pvar = MatrixXd::Constant(tsize, 1, 0);</div>
<div class="line"><a name="l00099"></a><span class="lineno">   99</span>&#160;        trueval = MatrixXd::Constant(tsize, 1, 0);</div>
<div class="line"><a name="l00100"></a><span class="lineno">  100</span>&#160;        parallel = <a class="code" href="classpgpr__parallel.html">pgpr_parallel</a>(myid);</div>
<div class="line"><a name="l00101"></a><span class="lineno">  101</span>&#160;        blocksize = floor(dsize/globalBlocks);</div>
<div class="line"><a name="l00102"></a><span class="lineno">  102</span>&#160;        MPI_Comm_rank(MPI_COMM_WORLD,&amp;myid);</div>
<div class="line"><a name="l00103"></a><span class="lineno">  103</span>&#160;        parallel.setRank(myid);</div>
<div class="line"><a name="l00104"></a><span class="lineno">  104</span>&#160;        <span class="comment">//Load local train data</span></div>
<div class="line"><a name="l00105"></a><span class="lineno">  105</span>&#160;        localTrainSamples = MatrixXd(dim + 1, 1);</div>
<div class="line"><a name="l00106"></a><span class="lineno">  106</span>&#160;        <span class="comment">/*When current machine is in the last serveral, bandwidth may not be as defined.</span></div>
<div class="line"><a name="l00107"></a><span class="lineno">  107</span>&#160;<span class="comment">         *For example, if band=1 &amp;&amp; myid=2 &amp;&amp; blk=3 then only one block in machine 3(myid=2)</span></div>
<div class="line"><a name="l00108"></a><span class="lineno">  108</span>&#160;<span class="comment">         *bandwidth should be redefined to be 0</span></div>
<div class="line"><a name="l00109"></a><span class="lineno">  109</span>&#160;<span class="comment">         */</span></div>
<div class="line"><a name="l00110"></a><span class="lineno">  110</span>&#160;        localband = MIN(bandwidth, globalBlocks - myid - 1);</div>
<div class="line"><a name="l00111"></a><span class="lineno">  111</span>&#160;        <span class="keywordtype">int</span> ds = loadLocalData(train, localTrainSamples, blocksize, myid, localband + 1);</div>
<div class="line"><a name="l00112"></a><span class="lineno">  112</span>&#160;        </div>
<div class="line"><a name="l00113"></a><span class="lineno">  113</span>&#160;        <span class="comment">//Load support sets</span></div>
<div class="line"><a name="l00114"></a><span class="lineno">  114</span>&#160;        supportSamples = MatrixXd(dim + 1, 1);</div>
<div class="line"><a name="l00115"></a><span class="lineno">  115</span>&#160;        <span class="keywordtype">int</span> ss = loadData(supset, supportSamples);</div>
<div class="line"><a name="l00116"></a><span class="lineno">  116</span>&#160;        </div>
<div class="line"><a name="l00117"></a><span class="lineno">  117</span>&#160;        <span class="comment">//Load test sets</span></div>
<div class="line"><a name="l00118"></a><span class="lineno">  118</span>&#160;        testClusters = <span class="keyword">new</span> vector&lt;VectorXd&gt;[blks];</div>
<div class="line"><a name="l00119"></a><span class="lineno">  119</span>&#160;        MatrixXd testSamples(dim + 1, 1);</div>
<div class="line"><a name="l00120"></a><span class="lineno">  120</span>&#160;        loadData(test,testSamples);</div>
<div class="line"><a name="l00121"></a><span class="lineno">  121</span>&#160;        cluster(testSamples);</div>
<div class="line"><a name="l00122"></a><span class="lineno">  122</span>&#160;        <span class="keywordtype">int</span> k = 0;</div>
<div class="line"><a name="l00123"></a><span class="lineno">  123</span>&#160;        <span class="keywordflow">for</span>(<span class="keywordtype">int</span> i = 0; i &lt; globalBlocks; i ++)</div>
<div class="line"><a name="l00124"></a><span class="lineno">  124</span>&#160;            <span class="keywordflow">for</span>(<span class="keywordtype">int</span> j = 0; j &lt; testClusters[i].size(); j ++)</div>
<div class="line"><a name="l00125"></a><span class="lineno">  125</span>&#160;                trueval(k++, 0) = testClusters[i][j][dim];</div>
<div class="line"><a name="l00126"></a><span class="lineno">  126</span>&#160;    }</div>
<div class="line"><a name="l00127"></a><span class="lineno">  127</span>&#160;</div>
<div class="line"><a name="l00128"></a><span class="lineno">  128</span>&#160;    ~<a class="code" href="classpgpr__plma.html">pgpr_plma</a>(){</div>
<div class="line"><a name="l00129"></a><span class="lineno">  129</span>&#160;        <span class="keyword">delete</span>[] testClusters;  </div>
<div class="line"><a name="l00130"></a><span class="lineno">  130</span>&#160;    }</div>
<div class="line"><a name="l00131"></a><span class="lineno">  131</span>&#160;    </div>
<div class="line"><a name="l00137"></a><span class="lineno"><a class="line" href="classpgpr__plma.html#a52c35beeae947892f4f22145a1d91953">  137</a></span>&#160;    <span class="keywordtype">void</span> <a class="code" href="classpgpr__plma.html#a52c35beeae947892f4f22145a1d91953">plma_regr</a>(){</div>
<div class="line"><a name="l00138"></a><span class="lineno">  138</span>&#160;</div>
<div class="line"><a name="l00139"></a><span class="lineno">  139</span>&#160;        <span class="comment">//timer conters </span></div>
<div class="line"><a name="l00140"></a><span class="lineno">  140</span>&#160;        <a class="code" href="classpgpr__timer.html">pgpr_timer</a> timer;</div>
<div class="line"><a name="l00141"></a><span class="lineno">  141</span>&#160;        </div>
<div class="line"><a name="l00142"></a><span class="lineno">  142</span>&#160;        <span class="comment">//MPI</span></div>
<div class="line"><a name="l00143"></a><span class="lineno">  143</span>&#160;        MPI_Status status;</div>
<div class="line"><a name="l00144"></a><span class="lineno">  144</span>&#160;        MPI_Request send_request,recv_request;</div>
<div class="line"><a name="l00145"></a><span class="lineno">  145</span>&#160;        </div>
<div class="line"><a name="l00146"></a><span class="lineno">  146</span>&#160;        <span class="comment">//K means kernel matrix, which is the same as Sigma used in the paper.</span></div>
<div class="line"><a name="l00147"></a><span class="lineno">  147</span>&#160;        <span class="comment">// Compute \K__{S (D_m \cup D_m^B)} </span></div>
<div class="line"><a name="l00148"></a><span class="lineno">  148</span>&#160;        timer.<a class="code" href="classpgpr__timer.html#ae30d8bfbf046764791f934d198f122b3">start</a>();</div>
<div class="line"><a name="l00149"></a><span class="lineno">  149</span>&#160;        MatrixXd K_SS;</div>
<div class="line"><a name="l00150"></a><span class="lineno">  150</span>&#160;        Matrix&lt;double, Dynamic, Dynamic, RowMajor&gt; K_DS;</div>
<div class="line"><a name="l00151"></a><span class="lineno">  151</span>&#160;        MatrixXd K_DD;</div>
<div class="line"><a name="l00152"></a><span class="lineno">  152</span>&#160;        <span class="comment">// K_DD_S = K_DD - K_DS * K_SS^-1 * K_SD</span></div>
<div class="line"><a name="l00153"></a><span class="lineno">  153</span>&#160;        MatrixXd K_DD_S;       </div>
<div class="line"><a name="l00154"></a><span class="lineno">  154</span>&#160;        MatrixXd tmp;</div>
<div class="line"><a name="l00155"></a><span class="lineno">  155</span>&#160;        <span class="comment">//Check Noise</span></div>
<div class="line"><a name="l00156"></a><span class="lineno">  156</span>&#160;        cov.se_ard(supportSamples, K_SS);    </div>
<div class="line"><a name="l00157"></a><span class="lineno">  157</span>&#160;        cov.se_ard(localTrainSamples, supportSamples, tmp);</div>
<div class="line"><a name="l00158"></a><span class="lineno">  158</span>&#160;        K_DS = tmp;</div>
<div class="line"><a name="l00159"></a><span class="lineno">  159</span>&#160;        cov.se_ard_n(localTrainSamples, K_DD);</div>
<div class="line"><a name="l00160"></a><span class="lineno">  160</span>&#160;</div>
<div class="line"><a name="l00161"></a><span class="lineno">  161</span>&#160;        MatrixXd K_DSTchol_SS; </div>
<div class="line"><a name="l00162"></a><span class="lineno">  162</span>&#160;        MatrixXd K_SD;</div>
<div class="line"><a name="l00163"></a><span class="lineno">  163</span>&#160;        MatrixXd K_DmDm_SDmB;</div>
<div class="line"><a name="l00164"></a><span class="lineno">  164</span>&#160;        LLT&lt;MatrixXd&gt; lltofK_DmDm_SDmB;</div>
<div class="line"><a name="l00165"></a><span class="lineno">  165</span>&#160;        MatrixXd K_DmDmB_S;</div>
<div class="line"><a name="l00166"></a><span class="lineno">  166</span>&#160;</div>
<div class="line"><a name="l00167"></a><span class="lineno">  167</span>&#160;        MatrixXd K_DmBDmB_S_inv;        </div>
<div class="line"><a name="l00168"></a><span class="lineno">  168</span>&#160;        MatrixXd dotK_SS;</div>
<div class="line"><a name="l00169"></a><span class="lineno">  169</span>&#160;        <span class="comment">//MK_DmS = K_DmS - K_DmDmB_S * K_DmBDmB_S^-1 * K_DmB_S</span></div>
<div class="line"><a name="l00170"></a><span class="lineno">  170</span>&#160;        MatrixXd MK_DmS;</div>
<div class="line"><a name="l00171"></a><span class="lineno">  171</span>&#160;</div>
<div class="line"><a name="l00172"></a><span class="lineno">  172</span>&#160;        </div>
<div class="line"><a name="l00173"></a><span class="lineno">  173</span>&#160;        <span class="comment">/* --------------- Precompute redudant part ----------------------*/</span></div>
<div class="line"><a name="l00174"></a><span class="lineno">  174</span>&#160;        <span class="comment">//        reS = K_DS * K_SS^-1</span></div>
<div class="line"><a name="l00175"></a><span class="lineno">  175</span>&#160;        MatrixXd reS;</div>
<div class="line"><a name="l00176"></a><span class="lineno">  176</span>&#160;        {</div>
<div class="line"><a name="l00177"></a><span class="lineno">  177</span>&#160;            MatrixXd K_SS_inv;</div>
<div class="line"><a name="l00178"></a><span class="lineno">  178</span>&#160;            K_SS_inv = K_SS.inverse();</div>
<div class="line"><a name="l00179"></a><span class="lineno">  179</span>&#160;            reS = K_DS * K_SS_inv;</div>
<div class="line"><a name="l00180"></a><span class="lineno">  180</span>&#160;        }</div>
<div class="line"><a name="l00181"></a><span class="lineno">  181</span>&#160;        <span class="comment">/*-------------- Precompute end -------------------------------*/</span></div>
<div class="line"><a name="l00182"></a><span class="lineno">  182</span>&#160;        </div>
<div class="line"><a name="l00183"></a><span class="lineno">  183</span>&#160;        <span class="comment">//TODO check noalias efficiency</span></div>
<div class="line"><a name="l00184"></a><span class="lineno">  184</span>&#160;        <span class="comment">//K_DD_S = K_DD - K_DS * KSS^-1 * K_SD</span></div>
<div class="line"><a name="l00185"></a><span class="lineno">  185</span>&#160;        K_DD_S = K_DD;</div>
<div class="line"><a name="l00186"></a><span class="lineno">  186</span>&#160;        K_DD_S.noalias() -= reS * K_DS.transpose();</div>
<div class="line"><a name="l00187"></a><span class="lineno">  187</span>&#160;        </div>
<div class="line"><a name="l00188"></a><span class="lineno">  188</span>&#160;        MatrixXd reK;</div>
<div class="line"><a name="l00189"></a><span class="lineno">  189</span>&#160;        </div>
<div class="line"><a name="l00190"></a><span class="lineno">  190</span>&#160;    </div>
<div class="line"><a name="l00192"></a><span class="lineno">  192</span>&#160;        <span class="comment">//</span></div>
<div class="line"><a name="l00193"></a><span class="lineno">  193</span>&#160;        <span class="comment">//   Compute dotK_SS</span></div>
<div class="line"><a name="l00194"></a><span class="lineno">  194</span>&#160;        <span class="comment">//</span></div>
<div class="line"><a name="l00196"></a><span class="lineno">  196</span>&#160;<span class="comment"></span>        <span class="comment">//Initialize: MK_DmS = K_DmS</span></div>
<div class="line"><a name="l00197"></a><span class="lineno">  197</span>&#160;        {</div>
<div class="line"><a name="l00198"></a><span class="lineno">  198</span>&#160;            <span class="comment">/* --------------- Precompute redudant part ----------------------*/</span></div>
<div class="line"><a name="l00199"></a><span class="lineno">  199</span>&#160;            <span class="comment">//        reK = K_DmDmB_S * K_DmBDmB_S^-1</span></div>
<div class="line"><a name="l00200"></a><span class="lineno">  200</span>&#160;            MatrixXd K_DmDmB_S;</div>
<div class="line"><a name="l00201"></a><span class="lineno">  201</span>&#160;            K_DmDm_SDmB = K_DD_S.topLeftCorner(blocksize, blocksize);</div>
<div class="line"><a name="l00202"></a><span class="lineno">  202</span>&#160;            <span class="keywordflow">if</span>(localband &gt;= 1){</div>
<div class="line"><a name="l00203"></a><span class="lineno">  203</span>&#160;                MatrixXd K_DmBDmB_S = K_DD_S.bottomRightCorner(localband * blocksize, localband * blocksize);</div>
<div class="line"><a name="l00204"></a><span class="lineno">  204</span>&#160;                K_DmDmB_S = K_DD_S.topRightCorner(blocksize, localband * blocksize);</div>
<div class="line"><a name="l00205"></a><span class="lineno">  205</span>&#160;                K_DmBDmB_S_inv = K_DmBDmB_S.inverse();</div>
<div class="line"><a name="l00206"></a><span class="lineno">  206</span>&#160;                reK = K_DmDmB_S * K_DmBDmB_S_inv;</div>
<div class="line"><a name="l00207"></a><span class="lineno">  207</span>&#160;            }</div>
<div class="line"><a name="l00208"></a><span class="lineno">  208</span>&#160;            <span class="comment">/*-------------- Precompute end -------------------------------*/</span></div>
<div class="line"><a name="l00209"></a><span class="lineno">  209</span>&#160;</div>
<div class="line"><a name="l00210"></a><span class="lineno">  210</span>&#160;            MK_DmS = K_DS.topRows(blocksize);</div>
<div class="line"><a name="l00211"></a><span class="lineno">  211</span>&#160;            </div>
<div class="line"><a name="l00212"></a><span class="lineno">  212</span>&#160;            <span class="comment">//check localband when compute K_DmBDmB_SDmB</span></div>
<div class="line"><a name="l00213"></a><span class="lineno">  213</span>&#160;            <span class="keywordflow">if</span>(localband &gt;= 1){</div>
<div class="line"><a name="l00214"></a><span class="lineno">  214</span>&#160;                MatrixXd K_DmBS = K_DS.bottomRows(localband * blocksize);</div>
<div class="line"><a name="l00215"></a><span class="lineno">  215</span>&#160;                K_DmDm_SDmB.noalias() -= reK * K_DmDmB_S.transpose();</div>
<div class="line"><a name="l00216"></a><span class="lineno">  216</span>&#160;                <span class="comment">//MK_DmS = K_DmS - K_DmDmB_S * K_DmBDmB_S^-1 * K_DmB_S</span></div>
<div class="line"><a name="l00217"></a><span class="lineno">  217</span>&#160;                MK_DmS.noalias() -= reK * K_DmBS;</div>
<div class="line"><a name="l00218"></a><span class="lineno">  218</span>&#160;            }</div>
<div class="line"><a name="l00219"></a><span class="lineno">  219</span>&#160;            </div>
<div class="line"><a name="l00220"></a><span class="lineno">  220</span>&#160;            <span class="comment">// dotK_SS = MK_DmS^T * K_DmDm_SDmB^-1 * MK_DmS</span></div>
<div class="line"><a name="l00221"></a><span class="lineno">  221</span>&#160;            tmp = MK_DmS.transpose();</div>
<div class="line"><a name="l00222"></a><span class="lineno">  222</span>&#160;            lltofK_DmDm_SDmB = K_DmDm_SDmB.llt();</div>
<div class="line"><a name="l00223"></a><span class="lineno">  223</span>&#160;            lltofK_DmDm_SDmB.solveInPlace(MK_DmS);</div>
<div class="line"><a name="l00224"></a><span class="lineno">  224</span>&#160;            dotK_SS = tmp * MK_DmS;</div>
<div class="line"><a name="l00225"></a><span class="lineno">  225</span>&#160;            parallel.sync();</div>
<div class="line"><a name="l00226"></a><span class="lineno">  226</span>&#160;        }</div>
<div class="line"><a name="l00228"></a><span class="lineno">  228</span>&#160;        <span class="comment">//</span></div>
<div class="line"><a name="l00229"></a><span class="lineno">  229</span>&#160;        <span class="comment">//    Compute dotY_S</span></div>
<div class="line"><a name="l00230"></a><span class="lineno">  230</span>&#160;        <span class="comment">//</span></div>
<div class="line"><a name="l00232"></a><span class="lineno">  232</span>&#160;<span class="comment"></span>            </div>
<div class="line"><a name="l00233"></a><span class="lineno">  233</span>&#160;</div>
<div class="line"><a name="l00234"></a><span class="lineno">  234</span>&#160;        MatrixXd dotY_S;</div>
<div class="line"><a name="l00235"></a><span class="lineno">  235</span>&#160;        <span class="comment">//MY_D = Y_D - K_DmDmB_S * KDmBDmB_S^-1 * YD</span></div>
<div class="line"><a name="l00236"></a><span class="lineno">  236</span>&#160;        VectorXd MY_Dm, Y_D;</div>
<div class="line"><a name="l00237"></a><span class="lineno">  237</span>&#160;        <span class="comment">//Initialize MY_D = Y_D = y_Dm - mean</span></div>
<div class="line"><a name="l00238"></a><span class="lineno">  238</span>&#160;        Y_D = localTrainSamples.row(dim);</div>
<div class="line"><a name="l00239"></a><span class="lineno">  239</span>&#160;        Y_D.noalias() -= VectorXd::Constant(localTrainSamples.cols(), mean);</div>
<div class="line"><a name="l00240"></a><span class="lineno">  240</span>&#160;        MY_Dm = Y_D.head(blocksize);</div>
<div class="line"><a name="l00241"></a><span class="lineno">  241</span>&#160;        <span class="keywordflow">if</span>(localband &gt;= 1){</div>
<div class="line"><a name="l00242"></a><span class="lineno">  242</span>&#160;            VectorXd Y_DmB;</div>
<div class="line"><a name="l00243"></a><span class="lineno">  243</span>&#160;            Y_DmB = Y_D.tail(localband * blocksize);</div>
<div class="line"><a name="l00244"></a><span class="lineno">  244</span>&#160;            MY_Dm.noalias() -= reK * Y_DmB;</div>
<div class="line"><a name="l00245"></a><span class="lineno">  245</span>&#160;        }</div>
<div class="line"><a name="l00246"></a><span class="lineno">  246</span>&#160;</div>
<div class="line"><a name="l00247"></a><span class="lineno">  247</span>&#160;        <span class="comment">//dotY_S = MK_DmS^T * K_DmDm_SDmB^-1 * MY_D</span></div>
<div class="line"><a name="l00248"></a><span class="lineno">  248</span>&#160;        lltofK_DmDm_SDmB.solveInPlace(MY_Dm);</div>
<div class="line"><a name="l00249"></a><span class="lineno">  249</span>&#160;        dotY_S.noalias() = tmp * MY_Dm;</div>
<div class="line"><a name="l00250"></a><span class="lineno">  250</span>&#160;        </div>
<div class="line"><a name="l00252"></a><span class="lineno">  252</span>&#160;        <span class="comment">//</span></div>
<div class="line"><a name="l00253"></a><span class="lineno">  253</span>&#160;        <span class="comment">//     Compute barK_DU = barK_DU_S + Q_DU</span></div>
<div class="line"><a name="l00254"></a><span class="lineno">  254</span>&#160;        <span class="comment">//     where Q_DU = K_DS * K_SS * K_SU</span></div>
<div class="line"><a name="l00255"></a><span class="lineno">  255</span>&#160;        <span class="comment">//</span></div>
<div class="line"><a name="l00257"></a><span class="lineno">  257</span>&#160;<span class="comment"></span>        Matrix&lt;double, Dynamic, Dynamic, RowMajor&gt; *barK_DU = <span class="keyword">new</span> Matrix&lt;double, Dynamic, Dynamic, RowMajor&gt;[globalBlocks];</div>
<div class="line"><a name="l00258"></a><span class="lineno">  258</span>&#160;        Matrix&lt;double, Dynamic, Dynamic, RowMajor&gt; *barK_DU_S = <span class="keyword">new</span> Matrix&lt;double, Dynamic, Dynamic, RowMajor&gt;[globalBlocks];</div>
<div class="line"><a name="l00259"></a><span class="lineno">  259</span>&#160;        MatrixXd *Q_DU= <span class="keyword">new</span> MatrixXd[globalBlocks];</div>
<div class="line"><a name="l00260"></a><span class="lineno">  260</span>&#160;        {</div>
<div class="line"><a name="l00261"></a><span class="lineno">  261</span>&#160;            <span class="keywordflow">for</span>(<span class="keywordtype">int</span> i = 0; i &lt; globalBlocks; i ++)</div>
<div class="line"><a name="l00262"></a><span class="lineno">  262</span>&#160;                barK_DU_S[i].resize(localTrainSamples.cols(), testClusters[i].size()); </div>
<div class="line"><a name="l00263"></a><span class="lineno">  263</span>&#160;            MatrixXd *K_SU = <span class="keyword">new</span> MatrixXd[globalBlocks];</div>
<div class="line"><a name="l00264"></a><span class="lineno">  264</span>&#160;            <span class="keywordflow">for</span>(<span class="keywordtype">int</span> i = 0; i &lt; globalBlocks; i ++){</div>
<div class="line"><a name="l00265"></a><span class="lineno">  265</span>&#160;                cov.se_ard(supportSamples, testClusters[i], tmp);</div>
<div class="line"><a name="l00266"></a><span class="lineno">  266</span>&#160;                K_SU[i] = tmp;</div>
<div class="line"><a name="l00267"></a><span class="lineno">  267</span>&#160;                Q_DU[i] = reS * K_SU[i];</div>
<div class="line"><a name="l00268"></a><span class="lineno">  268</span>&#160;            }</div>
<div class="line"><a name="l00269"></a><span class="lineno">  269</span>&#160;            <span class="comment">// There exist some uncessary computing blocks here</span></div>
<div class="line"><a name="l00270"></a><span class="lineno">  270</span>&#160;            <span class="comment">// ,which should not affact much. </span></div>
<div class="line"><a name="l00271"></a><span class="lineno">  271</span>&#160;            <span class="keywordflow">for</span>(<span class="keywordtype">int</span> i = MAX(0, myid - bandwidth); i &lt; MIN(myid + bandwidth + 1, globalBlocks); i ++){</div>
<div class="line"><a name="l00272"></a><span class="lineno">  272</span>&#160;                cov.se_ard(localTrainSamples, testClusters[i], tmp);</div>
<div class="line"><a name="l00273"></a><span class="lineno">  273</span>&#160;                barK_DU[i] = tmp;</div>
<div class="line"><a name="l00274"></a><span class="lineno">  274</span>&#160;</div>
<div class="line"><a name="l00275"></a><span class="lineno">  275</span>&#160;                barK_DU_S[i] = barK_DU[i] - Q_DU[i];</div>
<div class="line"><a name="l00276"></a><span class="lineno">  276</span>&#160;            }</div>
<div class="line"><a name="l00277"></a><span class="lineno">  277</span>&#160;            </div>
<div class="line"><a name="l00278"></a><span class="lineno">  278</span>&#160;            <span class="keyword">delete</span>[] K_SU;</div>
<div class="line"><a name="l00279"></a><span class="lineno">  279</span>&#160;        }</div>
<div class="line"><a name="l00280"></a><span class="lineno">  280</span>&#160;</div>
<div class="line"><a name="l00281"></a><span class="lineno">  281</span>&#160;        <span class="comment">/*-----------------------   Upper Triangular barK_DU_S------------------*/</span></div>
<div class="line"><a name="l00282"></a><span class="lineno">  282</span>&#160;        <span class="comment">// It can be computed diagonally by diagonally.</span></div>
<div class="line"><a name="l00283"></a><span class="lineno">  283</span>&#160;        <span class="comment">// Cur_diag is the global diagonal index working on.</span></div>
<div class="line"><a name="l00284"></a><span class="lineno">  284</span>&#160;        <span class="comment">// We only need to start from bandwidth, the blocks inside the bandwidth</span></div>
<div class="line"><a name="l00285"></a><span class="lineno">  285</span>&#160;        <span class="comment">// are already there. Note that the diag index starts from 0</span></div>
<div class="line"><a name="l00286"></a><span class="lineno">  286</span>&#160;        <span class="comment">// when m + B &lt; n: barK_DmUn_S = K_DmDmB_S * K_DmBDmB_S^-1 * barK_DmBUn_S</span></div>
<div class="line"><a name="l00287"></a><span class="lineno">  287</span>&#160;        <span class="keywordtype">int</span> cur_diag = bandwidth + 1;</div>
<div class="line"><a name="l00288"></a><span class="lineno">  288</span>&#160;        <span class="keywordflow">while</span>(cur_diag &lt; globalBlocks){</div>
<div class="line"><a name="l00289"></a><span class="lineno">  289</span>&#160;          </div>
<div class="line"><a name="l00290"></a><span class="lineno">  290</span>&#160;            <span class="comment">/*--------------- Each machine computes local corresponding block---------*/</span></div>
<div class="line"><a name="l00291"></a><span class="lineno">  291</span>&#160;            </div>
<div class="line"><a name="l00292"></a><span class="lineno">  292</span>&#160;            <span class="comment">// col is the global index of the test clusters/blocks. </span></div>
<div class="line"><a name="l00293"></a><span class="lineno">  293</span>&#160;            <span class="keywordtype">int</span> col = myid + cur_diag;</div>
<div class="line"><a name="l00294"></a><span class="lineno">  294</span>&#160;            </div>
<div class="line"><a name="l00295"></a><span class="lineno">  295</span>&#160;            <span class="keywordflow">if</span>( col &gt;= globalBlocks) <span class="keywordflow">break</span>;</div>
<div class="line"><a name="l00296"></a><span class="lineno">  296</span>&#160;            <span class="keywordflow">if</span>(bandwidth &gt; 0){</div>
<div class="line"><a name="l00297"></a><span class="lineno">  297</span>&#160;                MatrixXd barK_DmBUn_S;</div>
<div class="line"><a name="l00298"></a><span class="lineno">  298</span>&#160;                barK_DmBUn_S = barK_DU_S[col].bottomRows(bandwidth * blocksize);</div>
<div class="line"><a name="l00299"></a><span class="lineno">  299</span>&#160;                tmp = reK * barK_DmBUn_S;</div>
<div class="line"><a name="l00300"></a><span class="lineno">  300</span>&#160;            }<span class="keywordflow">else</span>{</div>
<div class="line"><a name="l00301"></a><span class="lineno">  301</span>&#160;                tmp = MatrixXd::Constant(blocksize, barK_DU_S[col].cols(), 0);</div>
<div class="line"><a name="l00302"></a><span class="lineno">  302</span>&#160;            }</div>
<div class="line"><a name="l00303"></a><span class="lineno">  303</span>&#160;            barK_DU_S[col].topRows(blocksize) = tmp;</div>
<div class="line"><a name="l00304"></a><span class="lineno">  304</span>&#160;            </div>
<div class="line"><a name="l00305"></a><span class="lineno">  305</span>&#160;            <span class="comment">/*---------------- Communicate the sharing blocks -----------------*/</span></div>
<div class="line"><a name="l00306"></a><span class="lineno">  306</span>&#160;            <span class="comment">//prepare for communication with the machine (myid - 1)</span></div>
<div class="line"><a name="l00307"></a><span class="lineno">  307</span>&#160;            <span class="keywordtype">int</span> updateBlocks = MIN(cur_diag - bandwidth, bandwidth);</div>
<div class="line"><a name="l00308"></a><span class="lineno">  308</span>&#160;            <span class="keywordflow">if</span>(myid &gt; 0){</div>
<div class="line"><a name="l00309"></a><span class="lineno">  309</span>&#160;                tmp = barK_DU_S[col].topRows(updateBlocks * blocksize);</div>
<div class="line"><a name="l00310"></a><span class="lineno">  310</span>&#160;                parallel.send_msg(tmp, myid, myid - 1);</div>
<div class="line"><a name="l00311"></a><span class="lineno">  311</span>&#160;            }</div>
<div class="line"><a name="l00312"></a><span class="lineno">  312</span>&#160;</div>
<div class="line"><a name="l00313"></a><span class="lineno">  313</span>&#160;            <span class="comment">//receive from the machine myid + 1</span></div>
<div class="line"><a name="l00314"></a><span class="lineno">  314</span>&#160;            <span class="comment">//some machine may be already idle now</span></div>
<div class="line"><a name="l00315"></a><span class="lineno">  315</span>&#160;            <span class="comment">// if current machine havn&#39;t finish the last col, then the next machine would communicate with current one</span></div>
<div class="line"><a name="l00316"></a><span class="lineno">  316</span>&#160;            <span class="keywordflow">if</span>(col &lt; globalBlocks - 1 ){</div>
<div class="line"><a name="l00317"></a><span class="lineno">  317</span>&#160;                tmp.resize(updateBlocks * blocksize, testClusters[col + 1].size());</div>
<div class="line"><a name="l00318"></a><span class="lineno">  318</span>&#160;                parallel.recv_msg(tmp, myid + 1, myid + 1);</div>
<div class="line"><a name="l00319"></a><span class="lineno">  319</span>&#160;                barK_DU_S[col + 1].block(blocksize, 0, updateBlocks * blocksize, testClusters[col + 1].size()) = tmp;</div>
<div class="line"><a name="l00320"></a><span class="lineno">  320</span>&#160;            }</div>
<div class="line"><a name="l00321"></a><span class="lineno">  321</span>&#160;            cur_diag ++;</div>
<div class="line"><a name="l00322"></a><span class="lineno">  322</span>&#160;        }</div>
<div class="line"><a name="l00323"></a><span class="lineno">  323</span>&#160;        parallel.sync();</div>
<div class="line"><a name="l00324"></a><span class="lineno">  324</span>&#160;</div>
<div class="line"><a name="l00325"></a><span class="lineno">  325</span>&#160;        <span class="comment">/*-----------------------   Lower Triangular barK_DU_S------------------*/</span></div>
<div class="line"><a name="l00326"></a><span class="lineno">  326</span>&#160;        <span class="comment">// Current implementation of computing lower triangular of barK_DU_S is by computing its </span></div>
<div class="line"><a name="l00327"></a><span class="lineno">  327</span>&#160;        <span class="comment">// tranpose barK_UD_S. In this way, we can save some computation and storage, but introduce</span></div>
<div class="line"><a name="l00328"></a><span class="lineno">  328</span>&#160;        <span class="comment">// the communication overhead. It may be possible to be improved, depending on the communication</span></div>
<div class="line"><a name="l00329"></a><span class="lineno">  329</span>&#160;        <span class="comment">// speed and memeory overhead. </span></div>
<div class="line"><a name="l00330"></a><span class="lineno">  330</span>&#160;        <span class="comment">// when m - B &gt; n: barK_DmUn_S = (barK_UnDm_S)^T</span></div>
<div class="line"><a name="l00331"></a><span class="lineno">  331</span>&#160;        <span class="comment">//                             = (barK_UnDnB_S * K_DnBDnB_S^-1 * barK_DnBDm_S)^T</span></div>
<div class="line"><a name="l00332"></a><span class="lineno">  332</span>&#160;        <span class="comment">// Note barK_DnBDm_S is unknown, compute it using the similiar idea as computing upper triangular</span></div>
<div class="line"><a name="l00333"></a><span class="lineno">  333</span>&#160;</div>
<div class="line"><a name="l00334"></a><span class="lineno">  334</span>&#160;        <span class="comment">/* -------------- Precompute redudant part --------------*/</span></div>
<div class="line"><a name="l00335"></a><span class="lineno">  335</span>&#160;        <span class="comment">//       reU = barK_UnDnB_S * K_DnBDnB_S^-1</span></div>
<div class="line"><a name="l00336"></a><span class="lineno">  336</span>&#160;        MatrixXd reU;</div>
<div class="line"><a name="l00337"></a><span class="lineno">  337</span>&#160;        <span class="keywordflow">if</span>(localband &gt;= 1){</div>
<div class="line"><a name="l00338"></a><span class="lineno">  338</span>&#160;            tmp = barK_DU_S[myid].bottomRows(localband * blocksize);</div>
<div class="line"><a name="l00339"></a><span class="lineno">  339</span>&#160;            reU = tmp.transpose() * K_DmBDmB_S_inv;</div>
<div class="line"><a name="l00340"></a><span class="lineno">  340</span>&#160;        }</div>
<div class="line"><a name="l00341"></a><span class="lineno">  341</span>&#160;        <span class="comment">/*-------------- Precompute end --------------*/</span></div>
<div class="line"><a name="l00342"></a><span class="lineno">  342</span>&#160;        cur_diag = bandwidth + 1;</div>
<div class="line"><a name="l00343"></a><span class="lineno">  343</span>&#160;        </div>
<div class="line"><a name="l00344"></a><span class="lineno">  344</span>&#160;        Matrix&lt;double, Dynamic, Dynamic, RowMajor&gt; barK_DnBDm_S;</div>
<div class="line"><a name="l00345"></a><span class="lineno">  345</span>&#160;        <span class="keywordflow">if</span>(localband == bandwidth)</div>
<div class="line"><a name="l00346"></a><span class="lineno">  346</span>&#160;            barK_DnBDm_S = K_DD_S.rightCols(blocksize);</div>
<div class="line"><a name="l00347"></a><span class="lineno">  347</span>&#160;</div>
<div class="line"><a name="l00348"></a><span class="lineno">  348</span>&#160;        <span class="keywordflow">while</span>( cur_diag &lt; globalBlocks){</div>
<div class="line"><a name="l00349"></a><span class="lineno">  349</span>&#160;            <span class="keywordtype">int</span> row, col;</div>
<div class="line"><a name="l00350"></a><span class="lineno">  350</span>&#160;            <span class="keywordtype">int</span> updateBlocks = bandwidth;</div>
<div class="line"><a name="l00351"></a><span class="lineno">  351</span>&#160;            {   <span class="comment">// Communication start: Sending the first bandwidth blocks of barK_DnDm_S to previous machine</span></div>
<div class="line"><a name="l00352"></a><span class="lineno">  352</span>&#160;                <span class="comment">// @detail:  to compute barK_DmDn_S, machine m need barK_DmBDn_S which would be sent from machine m + 1</span></div>
<div class="line"><a name="l00353"></a><span class="lineno">  353</span>&#160;                col= myid + cur_diag - 1;</div>
<div class="line"><a name="l00354"></a><span class="lineno">  354</span>&#160;                <span class="keywordflow">if</span>(col &lt; globalBlocks){<span class="comment">// other machines need do nothing</span></div>
<div class="line"><a name="l00355"></a><span class="lineno">  355</span>&#160;                    <span class="keywordflow">if</span>(myid &gt; 0 &amp;&amp; updateBlocks &gt; 0){<span class="comment">// sender </span></div>
<div class="line"><a name="l00356"></a><span class="lineno">  356</span>&#160;                        tmp = barK_DnBDm_S.topRows(updateBlocks * blocksize);</div>
<div class="line"><a name="l00357"></a><span class="lineno">  357</span>&#160;                        parallel.send_msg(tmp, myid, myid - 1);</div>
<div class="line"><a name="l00358"></a><span class="lineno">  358</span>&#160;                    }</div>
<div class="line"><a name="l00359"></a><span class="lineno">  359</span>&#160;                    <span class="keywordflow">if</span>(col  &lt;  globalBlocks - 1 &amp;&amp; updateBlocks &gt; 0){<span class="comment">// receiver</span></div>
<div class="line"><a name="l00360"></a><span class="lineno">  360</span>&#160;                        tmp.resize(updateBlocks * blocksize, blocksize);</div>
<div class="line"><a name="l00361"></a><span class="lineno">  361</span>&#160;                        parallel.recv_msg(tmp, myid + 1, myid + 1);</div>
<div class="line"><a name="l00362"></a><span class="lineno">  362</span>&#160;                        barK_DnBDm_S.bottomRows(updateBlocks * blocksize) = tmp;</div>
<div class="line"><a name="l00363"></a><span class="lineno">  363</span>&#160;                    }</div>
<div class="line"><a name="l00364"></a><span class="lineno">  364</span>&#160;                }</div>
<div class="line"><a name="l00365"></a><span class="lineno">  365</span>&#160;            }<span class="comment">// Communication end;</span></div>
<div class="line"><a name="l00366"></a><span class="lineno">  366</span>&#160;</div>
<div class="line"><a name="l00367"></a><span class="lineno">  367</span>&#160;</div>
<div class="line"><a name="l00368"></a><span class="lineno">  368</span>&#160;</div>
<div class="line"><a name="l00369"></a><span class="lineno">  369</span>&#160;            <span class="comment">// Computer barK_DmUn_S^T  in machine n, which is barK_UnDm_S</span></div>
<div class="line"><a name="l00370"></a><span class="lineno">  370</span>&#160;            <span class="keywordtype">int</span> upper_col = myid + cur_diag;</div>
<div class="line"><a name="l00371"></a><span class="lineno">  371</span>&#160;            <span class="keywordflow">if</span>(upper_col &lt; globalBlocks){</div>
<div class="line"><a name="l00372"></a><span class="lineno">  372</span>&#160;                <span class="keywordflow">if</span>(bandwidth &gt; 0){</div>
<div class="line"><a name="l00373"></a><span class="lineno">  373</span>&#160;                    tmp = barK_DnBDm_S.bottomRows(bandwidth * blocksize);</div>
<div class="line"><a name="l00374"></a><span class="lineno">  374</span>&#160;                    tmp = reU * tmp;</div>
<div class="line"><a name="l00375"></a><span class="lineno">  375</span>&#160;                }<span class="keywordflow">else</span>{</div>
<div class="line"><a name="l00376"></a><span class="lineno">  376</span>&#160;                    tmp = MatrixXd::Constant(testClusters[myid].size(), blocksize, 0);</div>
<div class="line"><a name="l00377"></a><span class="lineno">  377</span>&#160;                }</div>
<div class="line"><a name="l00378"></a><span class="lineno">  378</span>&#160;</div>
<div class="line"><a name="l00379"></a><span class="lineno">  379</span>&#160;                <span class="comment">// then send the result to the coressponding machine,which is transpose</span></div>
<div class="line"><a name="l00380"></a><span class="lineno">  380</span>&#160;                <span class="comment">// sending the resulting barK_DmUn_S from machine n to machine m</span></div>
<div class="line"><a name="l00381"></a><span class="lineno">  381</span>&#160;                parallel.send_msg(tmp, myid, upper_col);</div>
<div class="line"><a name="l00382"></a><span class="lineno">  382</span>&#160;                <span class="keywordflow">if</span>(bandwidth &gt; 0)</div>
<div class="line"><a name="l00383"></a><span class="lineno">  383</span>&#160;                    barK_DnBDm_S.block(0, 0, blocksize, blocksize) = reK * barK_DnBDm_S.bottomRows(bandwidth * blocksize);</div>
<div class="line"><a name="l00384"></a><span class="lineno">  384</span>&#160;            }</div>
<div class="line"><a name="l00385"></a><span class="lineno">  385</span>&#160;</div>
<div class="line"><a name="l00386"></a><span class="lineno">  386</span>&#160;            <span class="comment">//coressponding lower triangular col</span></div>
<div class="line"><a name="l00387"></a><span class="lineno">  387</span>&#160;            <span class="keywordtype">int</span> lower_col = myid - cur_diag;</div>
<div class="line"><a name="l00388"></a><span class="lineno">  388</span>&#160;            <span class="keywordflow">if</span>(lower_col&gt;= 0){</div>
<div class="line"><a name="l00389"></a><span class="lineno">  389</span>&#160;                <span class="comment">//the machines belong to the lower triangluar part should recive the message </span></div>
<div class="line"><a name="l00390"></a><span class="lineno">  390</span>&#160;                tmp.resize(testClusters[lower_col].size(), blocksize);</div>
<div class="line"><a name="l00391"></a><span class="lineno">  391</span>&#160;                parallel.recv_msg(tmp, lower_col, lower_col);</div>
<div class="line"><a name="l00392"></a><span class="lineno">  392</span>&#160;                barK_DU_S[lower_col].topRows(blocksize) = tmp.transpose();</div>
<div class="line"><a name="l00393"></a><span class="lineno">  393</span>&#160;            }</div>
<div class="line"><a name="l00394"></a><span class="lineno">  394</span>&#160;</div>
<div class="line"><a name="l00395"></a><span class="lineno">  395</span>&#160;            cur_diag ++;</div>
<div class="line"><a name="l00396"></a><span class="lineno">  396</span>&#160;            parallel.sync();</div>
<div class="line"><a name="l00397"></a><span class="lineno">  397</span>&#160;        }</div>
<div class="line"><a name="l00398"></a><span class="lineno">  398</span>&#160;</div>
<div class="line"><a name="l00399"></a><span class="lineno">  399</span>&#160;</div>
<div class="line"><a name="l00400"></a><span class="lineno">  400</span>&#160;        {<span class="comment">//till now, each machine should have the complete first row of barK_DU_S</span></div>
<div class="line"><a name="l00401"></a><span class="lineno">  401</span>&#160;        </div>
<div class="line"><a name="l00402"></a><span class="lineno">  402</span>&#160;            <span class="comment">// To fill the next localband rows of barK_DU_S, and reduce the communication overhead,</span></div>
<div class="line"><a name="l00403"></a><span class="lineno">  403</span>&#160;            <span class="comment">// the messages are exchanged between only neighbouring machines.</span></div>
<div class="line"><a name="l00404"></a><span class="lineno">  404</span>&#160;            <span class="comment">// for distance i, machine m send i + 1 row to machine m - 1.</span></div>
<div class="line"><a name="l00405"></a><span class="lineno">  405</span>&#160;            <span class="keywordtype">int</span> distance = 1;</div>
<div class="line"><a name="l00406"></a><span class="lineno">  406</span>&#160;            <span class="keywordflow">while</span>(distance &lt;= bandwidth){</div>
<div class="line"><a name="l00407"></a><span class="lineno">  407</span>&#160;</div>
<div class="line"><a name="l00408"></a><span class="lineno">  408</span>&#160;                <span class="keywordtype">double</span> *sendbuf = NULL; </div>
<div class="line"><a name="l00409"></a><span class="lineno">  409</span>&#160;                <span class="keywordtype">double</span> *recvbuf = NULL;</div>
<div class="line"><a name="l00410"></a><span class="lineno">  410</span>&#160;                </div>
<div class="line"><a name="l00411"></a><span class="lineno">  411</span>&#160;                <span class="keywordflow">if</span>(myid - bandwidth + distance &gt; 1 &amp;&amp; myid &lt; globalBlocks - distance + 1){</div>
<div class="line"><a name="l00412"></a><span class="lineno">  412</span>&#160;</div>
<div class="line"><a name="l00413"></a><span class="lineno">  413</span>&#160;                    <span class="keywordtype">int</span> cols = myid - bandwidth + distance - 1;</div>
<div class="line"><a name="l00414"></a><span class="lineno">  414</span>&#160;                    <span class="keywordtype">int</span> sendbufsize = 0;</div>
<div class="line"><a name="l00415"></a><span class="lineno">  415</span>&#160;                    <span class="keywordflow">for</span>(<span class="keywordtype">int</span> i = 0; i &lt; cols; i ++)</div>
<div class="line"><a name="l00416"></a><span class="lineno">  416</span>&#160;                        sendbufsize += blocksize * testClusters[i].size();           </div>
<div class="line"><a name="l00417"></a><span class="lineno">  417</span>&#160;                    sendbuf = <span class="keyword">new</span> <span class="keywordtype">double</span>[sendbufsize];</div>
<div class="line"><a name="l00418"></a><span class="lineno">  418</span>&#160;                    <span class="keywordtype">int</span> k = 0;</div>
<div class="line"><a name="l00419"></a><span class="lineno">  419</span>&#160;                    <span class="keywordflow">for</span>(<span class="keywordtype">int</span> col = 0; col &lt; cols; col ++){</div>
<div class="line"><a name="l00420"></a><span class="lineno">  420</span>&#160;                        <span class="keywordflow">for</span>(<span class="keywordtype">int</span> i = 0; i &lt; blocksize; i ++)</div>
<div class="line"><a name="l00421"></a><span class="lineno">  421</span>&#160;                            <span class="keywordflow">for</span>(<span class="keywordtype">int</span> j = 0; j &lt; testClusters[col].size(); j ++)</div>
<div class="line"><a name="l00422"></a><span class="lineno">  422</span>&#160;                                sendbuf[k++] = barK_DU_S[col](i + blocksize * (distance - 1), j);</div>
<div class="line"><a name="l00423"></a><span class="lineno">  423</span>&#160;                    }</div>
<div class="line"><a name="l00424"></a><span class="lineno">  424</span>&#160;                    MPI_Isend(sendbuf,sendbufsize, MPI_DOUBLE, myid - 1,myid, MPI_COMM_WORLD,&amp;send_request);</div>
<div class="line"><a name="l00425"></a><span class="lineno">  425</span>&#160;                    MPI_Wait(&amp;send_request, &amp;status);</div>
<div class="line"><a name="l00426"></a><span class="lineno">  426</span>&#160;                }</div>
<div class="line"><a name="l00427"></a><span class="lineno">  427</span>&#160;                </div>
<div class="line"><a name="l00428"></a><span class="lineno">  428</span>&#160;                <span class="keywordflow">if</span>(myid + distance - bandwidth &gt; 0 &amp;&amp; myid &lt; globalBlocks - distance ){</div>
<div class="line"><a name="l00429"></a><span class="lineno">  429</span>&#160;                    <span class="comment">//receiver</span></div>
<div class="line"><a name="l00430"></a><span class="lineno">  430</span>&#160;                    <span class="keywordtype">int</span> fromMachine = myid + 1;</div>
<div class="line"><a name="l00431"></a><span class="lineno">  431</span>&#160;                    <span class="keywordtype">int</span> recvbufsize = 0;</div>
<div class="line"><a name="l00432"></a><span class="lineno">  432</span>&#160;                    <span class="keywordtype">int</span> cols = myid + distance - bandwidth;</div>
<div class="line"><a name="l00433"></a><span class="lineno">  433</span>&#160;                    <span class="keywordflow">for</span>(<span class="keywordtype">int</span> i = 0; i &lt; cols; i ++)</div>
<div class="line"><a name="l00434"></a><span class="lineno">  434</span>&#160;                        recvbufsize += blocksize * testClusters[i].size();</div>
<div class="line"><a name="l00435"></a><span class="lineno">  435</span>&#160;                    recvbuf = <span class="keyword">new</span> <span class="keywordtype">double</span>[recvbufsize];</div>
<div class="line"><a name="l00436"></a><span class="lineno">  436</span>&#160;                    </div>
<div class="line"><a name="l00437"></a><span class="lineno">  437</span>&#160;                    MPI_Recv(recvbuf, recvbufsize, MPI_DOUBLE, fromMachine, fromMachine, MPI_COMM_WORLD, &amp;status);</div>
<div class="line"><a name="l00438"></a><span class="lineno">  438</span>&#160;                    </div>
<div class="line"><a name="l00439"></a><span class="lineno">  439</span>&#160;                    <span class="keywordtype">int</span> k = 0;</div>
<div class="line"><a name="l00440"></a><span class="lineno">  440</span>&#160;                    <span class="keywordflow">for</span>(<span class="keywordtype">int</span> col = 0; col &lt; cols ; col ++){</div>
<div class="line"><a name="l00441"></a><span class="lineno">  441</span>&#160;                        <span class="keywordflow">for</span>(<span class="keywordtype">int</span> i = 0; i &lt; blocksize; i ++)</div>
<div class="line"><a name="l00442"></a><span class="lineno">  442</span>&#160;                            <span class="keywordflow">for</span>(<span class="keywordtype">int</span> j = 0; j &lt; testClusters[col].size(); j ++)</div>
<div class="line"><a name="l00443"></a><span class="lineno">  443</span>&#160;                                barK_DU_S[col](i + blocksize * distance, j) = recvbuf[k++];</div>
<div class="line"><a name="l00444"></a><span class="lineno">  444</span>&#160;                    }</div>
<div class="line"><a name="l00445"></a><span class="lineno">  445</span>&#160;                }</div>
<div class="line"><a name="l00446"></a><span class="lineno">  446</span>&#160;                <span class="keywordflow">if</span>(sendbuf != NULL){ <span class="keyword">delete</span> sendbuf;sendbuf=0;}</div>
<div class="line"><a name="l00447"></a><span class="lineno">  447</span>&#160;                <span class="keywordflow">if</span>(recvbuf != NULL) {<span class="keyword">delete</span> recvbuf;recvbuf=0;}</div>
<div class="line"><a name="l00448"></a><span class="lineno">  448</span>&#160;                distance ++;</div>
<div class="line"><a name="l00449"></a><span class="lineno">  449</span>&#160;            }</div>
<div class="line"><a name="l00450"></a><span class="lineno">  450</span>&#160;</div>
<div class="line"><a name="l00451"></a><span class="lineno">  451</span>&#160;        }</div>
<div class="line"><a name="l00452"></a><span class="lineno">  452</span>&#160;        parallel.sync();</div>
<div class="line"><a name="l00453"></a><span class="lineno">  453</span>&#160;</div>
<div class="line"><a name="l00454"></a><span class="lineno">  454</span>&#160;        <span class="comment">//    computer barK_DU = barK_DU_S + Q_DU</span></div>
<div class="line"><a name="l00455"></a><span class="lineno">  455</span>&#160;        <span class="keywordflow">for</span>(<span class="keywordtype">int</span> i = 0; i &lt; globalBlocks; i ++)</div>
<div class="line"><a name="l00456"></a><span class="lineno">  456</span>&#160;            barK_DU[i] = barK_DU_S[i] + Q_DU[i];</div>
<div class="line"><a name="l00457"></a><span class="lineno">  457</span>&#160;        </div>
<div class="line"><a name="l00458"></a><span class="lineno">  458</span>&#160;        <span class="keyword">delete</span>[] barK_DU_S;</div>
<div class="line"><a name="l00459"></a><span class="lineno">  459</span>&#160;        <span class="keyword">delete</span>[]  Q_DU;</div>
<div class="line"><a name="l00460"></a><span class="lineno">  460</span>&#160;</div>
<div class="line"><a name="l00461"></a><span class="lineno">  461</span>&#160;        VectorXd testOffset(globalBlocks);</div>
<div class="line"><a name="l00462"></a><span class="lineno">  462</span>&#160;        testOffset[0] = 0;</div>
<div class="line"><a name="l00463"></a><span class="lineno">  463</span>&#160;        <span class="keywordflow">for</span>(<span class="keywordtype">int</span> i = 1; i &lt; globalBlocks; i ++)</div>
<div class="line"><a name="l00464"></a><span class="lineno">  464</span>&#160;            testOffset[i] = testOffset[i - 1] + testClusters[i - 1].size();</div>
<div class="line"><a name="l00465"></a><span class="lineno">  465</span>&#160;</div>
<div class="line"><a name="l00466"></a><span class="lineno">  466</span>&#160;</div>
<div class="line"><a name="l00467"></a><span class="lineno">  467</span>&#160;        <span class="comment">//  Compute global summary ddotK_SS</span></div>
<div class="line"><a name="l00468"></a><span class="lineno">  468</span>&#160;        MatrixXd ddotK_SS;</div>
<div class="line"><a name="l00469"></a><span class="lineno">  469</span>&#160;        parallel.MapReduce(dotK_SS, ddotK_SS, MPI_SUM, -1);</div>
<div class="line"><a name="l00470"></a><span class="lineno">  470</span>&#160;        ddotK_SS.noalias() += K_SS;</div>
<div class="line"><a name="l00471"></a><span class="lineno">  471</span>&#160;        </div>
<div class="line"><a name="l00472"></a><span class="lineno">  472</span>&#160;        <span class="comment">// Compute global summary ddotY_S</span></div>
<div class="line"><a name="l00473"></a><span class="lineno">  473</span>&#160;        MatrixXd ddotY_S;</div>
<div class="line"><a name="l00474"></a><span class="lineno">  474</span>&#160;        parallel.MapReduce(dotY_S, ddotY_S, MPI_SUM, -1);</div>
<div class="line"><a name="l00475"></a><span class="lineno">  475</span>&#160;        parallel.sync();</div>
<div class="line"><a name="l00476"></a><span class="lineno">  476</span>&#160;        </div>
<div class="line"><a name="l00477"></a><span class="lineno">  477</span>&#160;        <span class="comment">//global summary</span></div>
<div class="line"><a name="l00478"></a><span class="lineno">  478</span>&#160;        MatrixXd ddotY_U;</div>
<div class="line"><a name="l00479"></a><span class="lineno">  479</span>&#160;        MatrixXd ddotK_US;</div>
<div class="line"><a name="l00480"></a><span class="lineno">  480</span>&#160;        MatrixXd ddotK_UU;</div>
<div class="line"><a name="l00481"></a><span class="lineno">  481</span>&#160;        <span class="comment">// do prediction for each cluster/block</span></div>
<div class="line"><a name="l00482"></a><span class="lineno">  482</span>&#160;        <span class="comment">/*  Here the prediciton is done in parallel. So each machine would store the coressponding part of dotY_U,</span></div>
<div class="line"><a name="l00483"></a><span class="lineno">  483</span>&#160;<span class="comment">         *  dotK_US and dotK_UU. For example, </span></div>
<div class="line"><a name="l00484"></a><span class="lineno">  484</span>&#160;<span class="comment">         */</span></div>
<div class="line"><a name="l00485"></a><span class="lineno">  485</span>&#160;        <span class="keywordflow">for</span>(<span class="keywordtype">int</span> clusterid = 0; clusterid &lt; globalBlocks; clusterid ++){</div>
<div class="line"><a name="l00486"></a><span class="lineno">  486</span>&#160;            <span class="keywordtype">int</span> localtsize = testClusters[clusterid].size();</div>
<div class="line"><a name="l00487"></a><span class="lineno">  487</span>&#160;            <span class="keywordflow">if</span>(localtsize &lt;= 0 )</div>
<div class="line"><a name="l00488"></a><span class="lineno">  488</span>&#160;                <span class="keywordflow">continue</span>;</div>
<div class="line"><a name="l00489"></a><span class="lineno">  489</span>&#160;            MatrixXd dotY_U = MatrixXd::Constant(localtsize, 1, 0);</div>
<div class="line"><a name="l00490"></a><span class="lineno">  490</span>&#160;            MatrixXd dotK_US = MatrixXd::Constant(localtsize, dotK_SS.rows(), 0);</div>
<div class="line"><a name="l00491"></a><span class="lineno">  491</span>&#160;            <span class="comment">//Note: here we don&#39;t compute the whole matrix dotK_UU, but only diagonal part of dotK_UU.</span></div>
<div class="line"><a name="l00492"></a><span class="lineno">  492</span>&#160;            MatrixXd dotK_UU = VectorXd::Constant(localtsize, 1, 0);</div>
<div class="line"><a name="l00493"></a><span class="lineno">  493</span>&#160;            </div>
<div class="line"><a name="l00494"></a><span class="lineno">  494</span>&#160;            MatrixXd barK_DmU;</div>
<div class="line"><a name="l00495"></a><span class="lineno">  495</span>&#160;            barK_DmU = barK_DU[clusterid].topRows(blocksize);</div>
<div class="line"><a name="l00496"></a><span class="lineno">  496</span>&#160;            </div>
<div class="line"><a name="l00497"></a><span class="lineno">  497</span>&#160;            <span class="keywordflow">if</span>(localband &gt;= 1){</div>
<div class="line"><a name="l00498"></a><span class="lineno">  498</span>&#160;                barK_DmU.noalias() -= reK * barK_DU[clusterid].bottomRows(localband * blocksize);</div>
<div class="line"><a name="l00499"></a><span class="lineno">  499</span>&#160;            }</div>
<div class="line"><a name="l00500"></a><span class="lineno">  500</span>&#160;            tmp = barK_DmU.transpose();</div>
<div class="line"><a name="l00501"></a><span class="lineno">  501</span>&#160;            </div>
<div class="line"><a name="l00502"></a><span class="lineno">  502</span>&#160;            dotY_U.noalias() = tmp * MY_Dm;</div>
<div class="line"><a name="l00503"></a><span class="lineno">  503</span>&#160;</div>
<div class="line"><a name="l00504"></a><span class="lineno">  504</span>&#160;            dotK_US.noalias() = tmp * MK_DmS;</div>
<div class="line"><a name="l00505"></a><span class="lineno">  505</span>&#160;</div>
<div class="line"><a name="l00506"></a><span class="lineno">  506</span>&#160;            tmp = barK_DmU;</div>
<div class="line"><a name="l00507"></a><span class="lineno">  507</span>&#160;            lltofK_DmDm_SDmB.solveInPlace(barK_DmU);</div>
<div class="line"><a name="l00508"></a><span class="lineno">  508</span>&#160;            </div>
<div class="line"><a name="l00509"></a><span class="lineno">  509</span>&#160;            <span class="keywordflow">for</span>(<span class="keywordtype">int</span> i = 0; i &lt; localtsize; i ++){</div>
<div class="line"><a name="l00510"></a><span class="lineno">  510</span>&#160;                dotK_UU(i, 0) = barK_DmU.col(i).dot(tmp.col(i));</div>
<div class="line"><a name="l00511"></a><span class="lineno">  511</span>&#160;</div>
<div class="line"><a name="l00512"></a><span class="lineno">  512</span>&#160;            }</div>
<div class="line"><a name="l00513"></a><span class="lineno">  513</span>&#160;            parallel.MapReduce(dotY_U, ddotY_U, MPI_SUM, clusterid);</div>
<div class="line"><a name="l00514"></a><span class="lineno">  514</span>&#160;            parallel.MapReduce(dotK_US, ddotK_US, MPI_SUM, clusterid);</div>
<div class="line"><a name="l00515"></a><span class="lineno">  515</span>&#160;            parallel.MapReduce(dotK_UU, ddotK_UU, MPI_SUM, clusterid);</div>
<div class="line"><a name="l00516"></a><span class="lineno">  516</span>&#160;        }</div>
<div class="line"><a name="l00517"></a><span class="lineno">  517</span>&#160;        parallel.sync();</div>
<div class="line"><a name="l00518"></a><span class="lineno">  518</span>&#160;        <span class="keyword">delete</span>[] barK_DU;</div>
<div class="line"><a name="l00519"></a><span class="lineno">  519</span>&#160;        <span class="comment">//compute predicted mean and variance</span></div>
<div class="line"><a name="l00520"></a><span class="lineno">  520</span>&#160;        {</div>
<div class="line"><a name="l00521"></a><span class="lineno">  521</span>&#160;            <span class="comment">//mean</span></div>
<div class="line"><a name="l00522"></a><span class="lineno">  522</span>&#160;            </div>
<div class="line"><a name="l00523"></a><span class="lineno">  523</span>&#160;            <span class="keywordtype">int</span> offset = testOffset[myid];</div>
<div class="line"><a name="l00524"></a><span class="lineno">  524</span>&#160;            <span class="keywordtype">int</span> localsize = testClusters[myid].size();</div>
<div class="line"><a name="l00525"></a><span class="lineno">  525</span>&#160;            MatrixXd localpmu = pmu;</div>
<div class="line"><a name="l00526"></a><span class="lineno">  526</span>&#160;            MatrixXd localpvar = pvar;</div>
<div class="line"><a name="l00527"></a><span class="lineno">  527</span>&#160;            <span class="keywordflow">if</span>( localsize &gt; 0){</div>
<div class="line"><a name="l00528"></a><span class="lineno">  528</span>&#160;                LLT&lt;MatrixXd&gt; lltofddotK_SS = ddotK_SS.llt();</div>
<div class="line"><a name="l00529"></a><span class="lineno">  529</span>&#160;                lltofddotK_SS.solveInPlace(ddotY_S);</div>
<div class="line"><a name="l00530"></a><span class="lineno">  530</span>&#160;</div>
<div class="line"><a name="l00531"></a><span class="lineno">  531</span>&#160;                localpmu.block(offset, 0, localsize, 1).noalias() = ddotY_U + MatrixXd::Constant(localsize, 1, mean);</div>
<div class="line"><a name="l00532"></a><span class="lineno">  532</span>&#160;                localpmu.block(offset, 0, localsize, 1).noalias() -= ddotK_US * ddotY_S;</div>
<div class="line"><a name="l00533"></a><span class="lineno">  533</span>&#160;                </div>
<div class="line"><a name="l00534"></a><span class="lineno">  534</span>&#160;                <span class="comment">//variance</span></div>
<div class="line"><a name="l00535"></a><span class="lineno">  535</span>&#160;</div>
<div class="line"><a name="l00536"></a><span class="lineno">  536</span>&#160;                localpvar.block(offset, 0, localsize, 1)= MatrixXd::Constant(localsize, 1, cov.sig + cov.nos);</div>
<div class="line"><a name="l00537"></a><span class="lineno">  537</span>&#160;                tmp = ddotK_US.transpose();</div>
<div class="line"><a name="l00538"></a><span class="lineno">  538</span>&#160;                lltofddotK_SS.solveInPlace(tmp);</div>
<div class="line"><a name="l00539"></a><span class="lineno">  539</span>&#160;                </div>
<div class="line"><a name="l00540"></a><span class="lineno">  540</span>&#160;                <span class="keywordflow">for</span>(<span class="keywordtype">int</span> i = 0; i &lt; localsize; i ++){</div>
<div class="line"><a name="l00541"></a><span class="lineno">  541</span>&#160;                    localpvar(i + offset, 0) += ddotK_US.row(i).dot(tmp.col(i)) - ddotK_UU(i, 0);</div>
<div class="line"><a name="l00542"></a><span class="lineno">  542</span>&#160;                }</div>
<div class="line"><a name="l00543"></a><span class="lineno">  543</span>&#160;                </div>
<div class="line"><a name="l00544"></a><span class="lineno">  544</span>&#160;            }</div>
<div class="line"><a name="l00545"></a><span class="lineno">  545</span>&#160;            parallel.MapReduce(localpmu, pmu, MPI_SUM, 0);</div>
<div class="line"><a name="l00546"></a><span class="lineno">  546</span>&#160;            parallel.MapReduce(localpvar, pvar, MPI_SUM, 0);</div>
<div class="line"><a name="l00547"></a><span class="lineno">  547</span>&#160;        }</div>
<div class="line"><a name="l00548"></a><span class="lineno">  548</span>&#160;    }</div>
<div class="line"><a name="l00549"></a><span class="lineno">  549</span>&#160;    <span class="keywordtype">int</span> regress(){</div>
<div class="line"><a name="l00550"></a><span class="lineno">  550</span>&#160;        <a class="code" href="classpgpr__timer.html">pgpr_timer</a> timer;</div>
<div class="line"><a name="l00551"></a><span class="lineno">  551</span>&#160;        parallel.sync(); timer.<a class="code" href="classpgpr__timer.html#ae30d8bfbf046764791f934d198f122b3">start</a>();</div>
<div class="line"><a name="l00552"></a><span class="lineno">  552</span>&#160;        plma_regr();</div>
<div class="line"><a name="l00553"></a><span class="lineno">  553</span>&#160;        parallel.sync();</div>
<div class="line"><a name="l00554"></a><span class="lineno">  554</span>&#160;        elapsed = timer.<a class="code" href="classpgpr__timer.html#ac81e09244717a72ce57242e86aa6c200">end</a>();</div>
<div class="line"><a name="l00555"></a><span class="lineno">  555</span>&#160;        <span class="keywordflow">if</span>(myid == 0){</div>
<div class="line"><a name="l00556"></a><span class="lineno">  556</span>&#160;            rmse = getRmse(trueval.col(0), pmu.col(0));</div>
<div class="line"><a name="l00557"></a><span class="lineno">  557</span>&#160;            mnlp = getMnlp(trueval.col(0), pmu.col(0), pvar.col(0));</div>
<div class="line"><a name="l00558"></a><span class="lineno">  558</span>&#160;        }</div>
<div class="line"><a name="l00559"></a><span class="lineno">  559</span>&#160;        <span class="keywordflow">return</span> 0;</div>
<div class="line"><a name="l00560"></a><span class="lineno">  560</span>&#160;    }</div>
<div class="line"><a name="l00561"></a><span class="lineno">  561</span>&#160;</div>
<div class="line"><a name="l00562"></a><span class="lineno">  562</span>&#160;    <span class="keywordtype">void</span> outputRst(Char * output){</div>
<div class="line"><a name="l00563"></a><span class="lineno">  563</span>&#160;        FILE *fp;</div>
<div class="line"><a name="l00564"></a><span class="lineno">  564</span>&#160;        <span class="keywordtype">int</span> ts = pmu.size();</div>
<div class="line"><a name="l00565"></a><span class="lineno">  565</span>&#160;        fp = fopen(output, <span class="stringliteral">&quot;w&quot;</span>);</div>
<div class="line"><a name="l00566"></a><span class="lineno">  566</span>&#160;        <span class="keywordflow">if</span>(fp == NULL){</div>
<div class="line"><a name="l00567"></a><span class="lineno">  567</span>&#160;            <span class="keywordflow">throw</span>(<span class="stringliteral">&quot;Fail to open file\n&quot;</span>);</div>
<div class="line"><a name="l00568"></a><span class="lineno">  568</span>&#160;        }</div>
<div class="line"><a name="l00569"></a><span class="lineno">  569</span>&#160;        <span class="keywordflow">for</span>(<span class="keywordtype">int</span> i = 0; i &lt; ts; i++) {</div>
<div class="line"><a name="l00570"></a><span class="lineno">  570</span>&#160;            fprintf(fp,<span class="stringliteral">&quot;%.4f %.4f %.4f\n&quot;</span>,trueval(i,0), pmu(i, 0), pvar(i, 0));</div>
<div class="line"><a name="l00571"></a><span class="lineno">  571</span>&#160;        }</div>
<div class="line"><a name="l00572"></a><span class="lineno">  572</span>&#160;        pmsg(LEV_PRG, stdout, <span class="stringliteral">&quot; %.4f | %.4f | %.4f |\n&quot;</span>,elapsed,rmse, mnlp);</div>
<div class="line"><a name="l00573"></a><span class="lineno">  573</span>&#160;</div>
<div class="line"><a name="l00574"></a><span class="lineno">  574</span>&#160;        fclose(fp);</div>
<div class="line"><a name="l00575"></a><span class="lineno">  575</span>&#160;    }</div>
<div class="line"><a name="l00576"></a><span class="lineno">  576</span>&#160;};</div>
<div class="line"><a name="l00577"></a><span class="lineno">  577</span>&#160;<span class="preprocessor">#endif</span></div>
<div class="ttc" id="pgpr__chol_8h_html"><div class="ttname"><a href="pgpr__chol_8h.html">pgpr_chol.h</a></div><div class="ttdoc">This file provides Cholesky factorization and some related useful functions such as inverse...</div></div>
<div class="ttc" id="classpgpr__plma_html_a91e6492d35e220988b68d49cfe851d61"><div class="ttname"><a href="classpgpr__plma.html#a91e6492d35e220988b68d49cfe851d61">pgpr_plma::pgpr_plma</a></div><div class="ttdeci">pgpr_plma(Char *hypf, Char *train, Char *test, Char *supset, int band, int blks)</div><div class="ttdoc">Every machine loads the corresponding portion of data, and cluster the test data into nearest blocks...</div><div class="ttdef"><b>Definition:</b> pgpr_plma.h:87</div></div>
<div class="ttc" id="classpgpr__timer_html_ac81e09244717a72ce57242e86aa6c200"><div class="ttname"><a href="classpgpr__timer.html#ac81e09244717a72ce57242e86aa6c200">pgpr_timer::end</a></div><div class="ttdeci">Doub end()</div><div class="ttdoc">Stop a timer and compute the elapsed time in seconds. </div><div class="ttdef"><b>Definition:</b> pgpr_util.h:732</div></div>
<div class="ttc" id="classpgpr__plma_html"><div class="ttname"><a href="classpgpr__plma.html">pgpr_plma</a></div><div class="ttdoc">This class provides the regression function using PLMA Approximation. </div><div class="ttdef"><b>Definition:</b> pgpr_plma.h:26</div></div>
<div class="ttc" id="pgpr__type_8h_html"><div class="ttname"><a href="pgpr__type_8h.html">pgpr_type.h</a></div><div class="ttdoc">This file provides important macros, templates, basic data types (e.g., vector, matrix). </div></div>
<div class="ttc" id="classpgpr__timer_html_ae30d8bfbf046764791f934d198f122b3"><div class="ttname"><a href="classpgpr__timer.html#ae30d8bfbf046764791f934d198f122b3">pgpr_timer::start</a></div><div class="ttdeci">void start()</div><div class="ttdoc">Start a timer. </div><div class="ttdef"><b>Definition:</b> pgpr_util.h:715</div></div>
<div class="ttc" id="classpgpr__plma_html_a52c35beeae947892f4f22145a1d91953"><div class="ttname"><a href="classpgpr__plma.html#a52c35beeae947892f4f22145a1d91953">pgpr_plma::plma_regr</a></div><div class="ttdeci">void plma_regr()</div><div class="ttdoc">main funciton for LMA regression </div><div class="ttdef"><b>Definition:</b> pgpr_plma.h:137</div></div>
<div class="ttc" id="pgpr__cov_8h_html"><div class="ttname"><a href="pgpr__cov_8h.html">pgpr_cov.h</a></div><div class="ttdoc">This file provides the covariance class: pgpr_cov, which compute the covariance matrix. </div></div>
<div class="ttc" id="classpgpr__parallel_html"><div class="ttname"><a href="classpgpr__parallel.html">pgpr_parallel</a></div><div class="ttdoc">the class provides the MPICH interface to commmunicate among the machines </div><div class="ttdef"><b>Definition:</b> pgpr_parallel.h:24</div></div>
<div class="ttc" id="classpgpr__timer_html"><div class="ttname"><a href="classpgpr__timer.html">pgpr_timer</a></div><div class="ttdoc">This timer class can provide real-time measure (in seconds) incurred by a block of running program...</div><div class="ttdef"><b>Definition:</b> pgpr_util.h:702</div></div>
<div class="ttc" id="classpgpr__cov_html"><div class="ttname"><a href="classpgpr__cov.html">pgpr_cov</a></div><div class="ttdoc">The pgpr_cov class provides the informaiton of covariance. </div><div class="ttdef"><b>Definition:</b> pgpr_cov.h:12</div></div>
</div><!-- fragment --></div><!-- contents -->
<!-- start footer part -->
<hr class="footer"/><address class="footer"><small>
Generated on Mon Jan 5 2015 14:32:10 for Parallel Gaussian Process Regression by &#160;<a href="http://www.doxygen.org/index.html">
<img class="footer" src="doxygen.png" alt="doxygen"/>
</a> 1.8.7
</small></address>
</body>
</html>
